-
Notifications
You must be signed in to change notification settings - Fork 358
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(#1225): create iob tags from record spans #1226
Conversation
@dcfidalgo Any idea about how to define as read-only the |
8d5fdec
to
54a5588
Compare
Finally I didn't found a way to keep text and tokens inmutables. So the tokens/chars map will be dynamically computed every time. I've include a cache resolution to avoid extra computations. Take a look @dcfidalgo |
Codecov Report
@@ Coverage Diff @@
## master #1226 +/- ##
==========================================
- Coverage 94.88% 94.84% -0.04%
==========================================
Files 127 127
Lines 5391 5449 +58
==========================================
+ Hits 5115 5168 +53
- Misses 276 281 +5
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
54a5588
to
c0b5900
Compare
@frascuchon Have a look at c0b5900 for making text and tokens immutable. Maybe we could move the "immutability" of |
* feat(#1225): create iob tags from record spans * test: add tests * refactor: dynamic tokens map with text/tokens mutability * chore: naming * feat: make text and tokens immutable * chore: adapt to inmutable text and tokens * test: fix tests * test: fixing tests Co-authored-by: dcfidalgo <david@recogn.ai> (cherry picked from commit 07b895d)
* feat(#1225): create iob tags from record spans * test: add tests * refactor: dynamic tokens map with text/tokens mutability * chore: naming * feat: make text and tokens immutable * chore: adapt to inmutable text and tokens * test: fix tests * test: fixing tests Co-authored-by: dcfidalgo <david@recogn.ai> (cherry picked from commit 07b895d)
* feat(#1225): create iob tags from record spans * test: add tests * refactor: dynamic tokens map with text/tokens mutability * chore: naming * feat: make text and tokens immutable * chore: adapt to inmutable text and tokens * test: fix tests * test: fixing tests Co-authored-by: dcfidalgo <david@recogn.ai> (cherry picked from commit 07b895d)
* feat(#1225): create iob tags from record spans * test: add tests * refactor: dynamic tokens map with text/tokens mutability * chore: naming * feat: make text and tokens immutable * chore: adapt to inmutable text and tokens * test: fix tests * test: fixing tests Co-authored-by: dcfidalgo <david@recogn.ai> (cherry picked from commit 07b895d)
In this PR we include client functionality for generate iob tags from record spans definitions. This will help for generate training dataset for huggingface models training
See #1225